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Abstract 

The purpose of this paper is to present an evaluation of a school reform project in 
one kindergarten through fifth grade elementary school to demonstrate how triangulation 
of qualitative and quantitative evaluation approaches can enhance understanding of a 
reform effort. Each methodology was conducted by a separate evaluator who was 
trained, experienced, and had expertise in the approach. Considered together, data from 
both approaches provided a broader understanding of the impacts of the reform in 
schools. Neither data set was used simply to gain greater understanding of the other 
data set, but instead, each data set was used to understand different aspects of the 
implementation of the reform, 



Introduction 

The American public and educators feel that there is a critical need for reform in public 
schools. This is not a new idea since each of the past three decades has spawned a 
new surge of reform to address problems identified in schools relative to emerging 
national and international concerns. The motivations for current school reform are as 
numerous as the definitions of reform. Motivations are economic, political, professional, 
and academic addressing such issues as empowerment (of teachers, students, parents), 
equity, social justice, social and cultural problems (drugs, teen pregnancy, single-parent 
families, etc.), expertise of teachers, quality of curriculum, quality and curriculum in 
teacher education, and literacy of students for the 21st century (Elmore, 1991; Goodlad, 
1991; Hess, 1992; Bacharach, 1990). 

Most school reform has taken place in isolated classrooms, schools, or school 
systems and the changes, if retained by those who pilot them, are rarely disseminated 
to the broader educational system (Elmore, 1991). Even broader and more longitudinal 
attempts at educational reform like Project 2061 (AAAS, 1989) to enhance scientific 
literacy of all American students for the 21st century will eventually result in a central 
curriculum {Science for all Americans) being implemented differently in different contexts 
across the nation. Since most public school reform is taking place in isolation, the key 
role that educational evaluators at these sites must play for the larger educational 
community is to share understandings about reform efforts as they occur within particular 
contexts. Generalizability or not (from the perspective of readers of educational 
evaluations of school reform) of findings cannot be determined without detailing contexts 



within which the reform is attempted. Because of the cultural, ethnic, economic, and 
political diversity in the nation, one single approach to school reform is impractical. 
Educators at each site must determine what reforms are necessary to make the teaching- 
learning process the best it can be for their school. 

School reform is not easy and it cannot be accomplished quickly. Funding for most 
school reform is short and policy-makers and funders typically want evaluation data which 
demonstrate effectiveness or not of a reform process within a year or two. Determination 
of "success" of school reform within such narrow time limits may be dysfunctional in that 
changes in student knowledge may not be demonstrated with current standardized test 
measures immediately, if at all. Many changes have been advocated in the name of 
alternative assessment, performance assessment, portfolio assessment and authentic 
assessment of student learning, but these approaches have not been well developed or 
tested across contexts. Meanwhile, most states have increased their use of standardized 
testing as a way of monitoring performance of local schools in the past decade (Elmore, 
1991). It is critical to collect process evaluation data as well as student achievement data 
to determine overall impact of school reform. 

The Community of Learners for Advancing Student Success Project 
Through a faculty collaborative process which was developed at Fowler Drive 
Elementary School beginning in 1987, the faculty voted in February of 1989 to 
reorganize the instructional delivery system in the school (the first year they developed 
"thematic" units and the last two years they developed "interdisciplinary" units), 
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implement a "family structure" within the school, increase parents* involvement with 
their childrens* learning, and implement an Extended Literacy Instruction (ELI) 
program. The original goal of the Community of Learners for Advancing Student 
Success (CLASS) Project was to enhance student learning and attitudes through 
interdisciplinary instruction and the creation of smaller, more stable family units within 
the school. The smaller units were proposed to facilitate communication, parent 
involvement and teachers' instructional decision-making process in the large school. 
The school reform project at Fowler Drive Elementary School began in summer, 1989 
and ended in summer, 1992. 

A major limitation of the evaluation of this project was the number of innovative 
things going on in the school which were not a part of the project. The year prior ;o 
the implementation of the project, many of the faculty had decided to implement a 
"whole language" learning approach using reading and writing workshop. Throughout 
the first year of the project, many teachers were confused about the relationship or not 
of whole language to the project. During the second year of the project, the school 
decided to be one of the pilot sites for the New York City Ballet curriculum. The 
curriculum provides information about the art of ballet and encourages a learning 
process that draws connections between areas of education and experiences in the 
child's life. The theme of the cuniculum is "Storytelling Through Dance." All fourth 
grade students were involved with this curriculum during the second year of the 
project and fifth grade students were involved with it the third year of the project. 
During the third year, some teachers were involved in a Foxfire workshop and adopted 



that philosophy for instruction. The notion of student directed curriculum resulted in a 
few of the teachers being opposed to pretest/posttest student assessment used in the 
project and caused them to question the whole school curriculum approach. 

Evaluation of the effect of the school reform project was very difficult because of 
the large number of other curricular changes that were taking place in the school at 
the same time. Analysis of qualitative data provided an understanding of the effect of 
different components of the project from parents' and teachers' perspectives. One 
teacher's comment that was representative of teachers' reflections at the end of the 
project was, "I have experienced so many positive things. But I don't know if the 
positive things Tve experienced are teacher sharing or because of the project." 
Another teacher said, "It's hard to put your finger on what the project is because there 
is so much going on. The problem is that every teacher does their own thing and 
picks the approach that they like at that time." 

Sgmp le 

Educational reform is not easy, quick, or predictable. School reform takes place 
within the larger culture of a community, thus, changes in the community and school 
system impact schools. During the summer between the first and second years of the 
project, the county school system underwent major redistricting. As a result of this, 
the student population at Fowler Drive Elementary School dropped from approximately 
870 to 490. The teaching staff was reduced from 55 teachers and 6 part-time 
teachers to 35 teachers- This resulted in physical plant changes in the school by the 



elimination of instruction in trailers external to the main school building which had been 
used during the first year of the project. 

Fowler Drive Elementary School is a kindergarten through 5th grade school in 
Athens, Georgia The socio-economic level of students ranged from middle class to 
lower class. Sixty-six percent of the students were eligible for free lunch the first year 
of the project and sixty-seven percent were eligible the second year of the project. 
There is a high student transient rate with an approximate 25% turn-over rate every 
year. 

During the 1991-92 school year, the final year of the project, Fowler Drive 
served approximately 484 students with a staff of 25 teachers, 1 principal, 1 assistant 
principal, a media specialist, and a half-time project director. The project had a full- 
time project director for the first two years. The 1992-93 school year has brought 
further changes which might affect continuation of the reform effort. The principal was 
transferred to a middle school in the same county. The project director assumed a 
position at the county central office. The media specialist who served as a part-time 
project assistant is no longer in the county. A few new teachers have joined the staff 
and some of the teachers who were a part of the reform effort have moved. It is 
interesting that in this case, as in most evaluations of school reform, there is no plan 
for continued evaluation of the impact of the reform beyond the funded years to see 
whether and how teachers continue to use what they have developed and change as 
the context changes. 
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Methodology 

A triangulated evaluation design of qualitative and quantitative approaches was 
used. Qualitative data were collected to determine the impact of the project on 
teachers' attitudes, parents' attitudes, and to describe the process. Quantitative data 
were collected to determine the impact of the project on student learning in basic 
skills, student reading ability, and student attitudes. Triangulation of investigators, data 
collection methods, and data sources were used. 

Bogdan and Biklen (1992) assert that researchers trying to combine good 
quantitative design and good qualitative design have a difficult time pulling it off, and 
rather than producing a superior hybrid, usually produce a piece of research that does 
not meet the criteria for good work in either approach. For this evaluation, a 
qualitative and a quantitative evaluator collected, analyzed, and interpreted data which 
addressed different objectives. The qualitative evaluator used student, parent, teacher, 
and administrator interview and open-ended questionnaire data to provide a 
description of the process used in the project and get participants' perspectives about 
different project components. She took observational fieldnotes during visits to the 
school throughout the three years. Qualitative data were analyzed using the constant 
comparative method of analysis (Strauss, 1987). Categories emerged from participant 
data and relationships among categories were examined to explain the impact of the 
project. 

Quantitative analysis of student achievement data provided information about the 
effects of the project on student learning as measured by standardized tests used in 
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the county and state. Changes in state testing requirements affected grade levels and 
types of data which were available for the third year of the project. The only 
standardized tests which were required for all Georgia students in elementary schools 
were the mathematics and reading tests of the Iowa Tests of Basic Skills {ITBS) for 
grades three and five for the final project year. For the first two years of the project, 
ITBS, COGAT, and the Georgia Criterion-Referenced Tests {CRT) scores were available 
for all children in the school. 

Different instruments were used to measure teachers' and students' attitude 
changes during the project. The Hall's Stages of Concern Questionnaire (SoC) was 
administered to teachers four times during the project. The focus for the questionnaire 
was thematic instruction for the first year and interdisciplinary instruction for the last 
two years of the project. There are seven stages of concern about which questions 
are focused in the questionnaire: awareness, informational, personal, management, 
consequence, collaboration, and refocusing. The Arlin Hills Attitude Toward Learning 
Process Survey (Arlin, 1976) was administered to all students in the school each year 
of the project. The instrument was a verbal/cartoon format. The student survey 
includes questions about use of time, teacher directiveness, contexts for learning, 
flexibility in the classroom, amount of homework, rates of learning, and peer 
collaboration in learning. 

Two instruments were used to measure changes in student attitude toward self 
and others. The Breakover Student Sense of Academic Futility Questionnaire was 
used for the first two years and on the suggestion of the on-site review team in 
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August, 1991, the Stanley Coopersmith Self-Esteem Inventory (SEI) (Coopersmith, 
1984) replaced the Bookover and was administered to students in fall, 1991 and 
spring, 1992. 

"Hiroughout the project, teachers struggled with the problem of student 
assessment. Many teachers who were totally committed to the "whole language" 
approach and interdisciplinary instruction felt that standardized tests did not assess 
the types of learning in which children were engaged. Portfolio assessment with a 
focus on student selection of worl< and development of self-assessment strategies was 
introduced and implemented in the second year of the project, but teachers 
abandoned it because most felt it was too time consuming. A few teachers 
maintained a portfolio approach of saving teacher-selected student work in files during 
the third project year. Teacher-constructed pretest/posttest assessment for each 
teacher-developed interdisciplinary unit was used the third year, but the quality of tests 
was inconsistent. Further, a few teachers who had taken a Foxfire approach 
workshop felt strongly that students should be actively involved in selecting the 
curriculum and thus felt that a pretest/posttest approach to student assessment was 
inappropriate. Some teachers worked with representatives from the Educational 
Testing Service to develop performance assessment for science objectives and two 
performance tests were administered to selected classes of students, but there was no 
scoring system in place to help evaluate performance. During the final focus-group 
interviews with teachers at the end of the 1991-92 academic year, almost all teachers 
said that they wanted student assessment strategies to be the focus of teacher staff- 
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development for the 1992-93 school year. Teachers want to investigate different 
approaches and try to find one which they feei matches the types of instruction that 
are being used at Fowler Drive. 

Qualitative Finding s 

Constas (1992) suggests that qualitative evaluations of programs may receive the 
names of their categories directly from programmatic objectives. The CLASS Project 
had specific objectives, and each of those objectives was best evaluated either by 
qualitative or quantitative data collection and analysis. The categories for qualitative 
data, however, were not dictated by the objectives, but rather emerged from narrative 
interview and open-ended questionnaire data from teachers, parents, and 
administrators. 

The Process , Teachers at Fowler Drive Elementary School believe that school 
reform takes time, that it is very complex, and that it is never ending. Teachers said 
that the process of determining the focus for school reform and trying different 
strategies and making changes when they felt things were not working was the most 
important thing which could be exported to other schools or school systems (See Fig. 
1). They asserted that adoption of the process would be much more likely to result in 
needed reform in other schools than simple adoption of completed teacher- 
constructed interdisciplinary units. A comment written by one teacher was repeated in 
different ways by almost every teacher in open-ended questionnaires or interviews: 
'They need to understand the PROCESS we went through. They need to understand 
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that there's never a final 'product' because knowledge and education is ever 
changing." Teachers felt that they were on the "cutting edge" of educational 
innovation in the nation. They felt that a critical part of this process was their 
willingness to "take a risk" and try something new, evaluate its effectiveness with their 
students, and be able to discard everything that did not work and examine new 
approaches for adoption. In the final open-ended questionnaire, teachers were asked 
what the most positive aspect of the CLASS Project was for them personally or 
professionally and 10 teachers said professional growth experiences, 9 teachers said 
the opportunity for planning with teachers across grade levels, 7 teachers said the 
integrated interdisciplinary focus for instruction, and 4 said the enhanced faculty 
relationships. These responses fit into the two larger categories which emerged 
across all teacher data: 1) importance of collaboration, and 2) increasing knowledge 
about instructional strategies. 

Although teachers felt that the process of reform is important, they often felt 
isolated from teachers at other schools within their own school system. One teacher 
said that negative encounters with those outside teachers were more frequent in the 
first two years, but that in the last year, teachers were more interested in finding out 
what had been implemented at Fowler Drive and how it had impacted teachers and 
students. One teacher characterized the negative perceptions of outsiders: "Teachers 
at other schools look down on us at Fowler Drive. They see us as real different 
When we got o meetings in the county, teachers don't listen to what we say when they 
find out we're from Fowler Drive. You don't want to tell people you're from Fowler." 
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Time. Multiple Changes, and Paperwork. Teachers felt that they were exposed to 
educaticnal innovation in a way that had never been before and would not have been 
without the CLASS Project. They felt that this was the most important part of the 
project. At the same time, this process often caused problems. Across the three 
years, teachers asserted that the amount of time required by the project, large number 
of changes being introduced at once, and amount of papenwork were the most 
negative aspects of the project. Almost all teachers in the final interview said that their 
school reform effort included too many components. Teachers' suggestions to 
schools thinking of adopting their reform was to select only one aspect to change 
initially, have intensive staff development, and then implement and evaluate only that 
change. Consider additional school structure and curricular changes when the bulk of 
work for the first change is completed. When 1 asked teachers which component they 
would recommend first to adopting schools, they could not identify one without 
asserting that it could not be effectively implemented without other changes. One 
teacher said, "That's a tough question because you can't change instructional 
strategies, school structure, or student assessment in isolation. It all affects the other. 
I guess that's the circular thing we got into." Another teacher said, "It's ironic that the 
most positive parts of the process have also been the most negative things in the 
project. Time it takes was time away from classes or planning for classes often. We 
were excited about change, but there was always change and we never felt we 
completed anything. Planning in teams often resulted in insufficient planning with 
grade levels. We often had the feeling that nothing is ever done, but that is a part of 
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change." 

Data collected on the Stages of Concern Questionnaire indicated that teachers' 
greatest concern throughout the three years of the project focused on issues related 
to management, including time demands, scheduling, organization, and management 
of the process. This concern did not diminish over the course of the project. Data 
from this instrument certainly reflect statements by teachers across the three years in 
interviews and open-ended questionnaires. 

Family Structure - Belonging and Competition . Teachers felt strongly that the 
family structure component was very effective when the school had close to 900 
students and many classes conducted outside of the school building in trailers. 
Teachers felt that the most important outcome was that students had a sense of 
belonging that they did not have in the larger atmosphere. A concern of some 
teachers was that different family units within the school increased a competitive spirit 
among students from different groups, and teachers felt that this was not positive. 

Teachers felt as strongly that, the same family structure was ineffective, and 
perhaps even had a negative effect after the school population was reduced by half. 
Three areas of concern emerged relative to the family structure with the smaller 
population: inability to move problem children to a different family, insufficient 
resources, and lack of group diversity over years for proper student social 
development. Teacher concerns emerged about the inability to split problem groups 
of children or to move children when the parents had difficulties with the teachers in 
one unit. Some teachers felt that in the smaller school population, students did not 
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get a chance to develop socially because they stayed with the same children over 
multiple years and they did not have a chance to develop and change roles. Teachers 
felt the family structure made it very difficult to use resources effectively because all 
teachers within a family were basically teaching the same curriculum and needed the 
same resources. Teachers said that they would suggest that teachers in any large 
school adopt the family structure, but that in smaller schools it should not be 
implemented. 

Parent Involvement. The first year of the project, Saturday workshops were 
conducted on topics of interest to parents about their childrens' learning. Childcare 
was provided for children during the parent workshops. While the low numbers of 
parents who attended the sessions judged their quality to be superior, these 
workshops were ineffective in increasing the number of parents involved in the 
educational process of their children. Teachers decided to abandon this format for 
parent involvement and develop some other strategy. 

Teachers decided that integrating the interdisciplinary instructional strategy and 
student demonstrations of learning with parent involvement might increase parental 
interest in their childrens' learning. During the last two project years, there were four 
day-long "whole school" educational events to which parents were invited. These were 
interdisciplinary and cross-grade level events which focused on a single topic 
developed by students throughout the quarter. Teachers who are "specialists" in the 
school (music, art, physical education) had primary responsibility for structuring the 
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events. For each event there were teacher/student presentations, student 
presentations, presentations by invited community guests and speakers, displays, and 
short optional parent workshops. Parents could see their children present their 
learning in a variety of formats including some in which parents participated. The 
number of participating parents increased steadily through the final year of the project 
from 26 parents to 127 parents. Teachers asserted that some parents had felt 
intimidated at the Saturday workshops which were presentations by "experts" in an 
area of child development, but that they were quite relaxed and interactive at the 
whole-school events held on a weekday while children were in school. Some teachers 
were concerned that parents of students with problems were not attracted by the 
Saturday workshops or the whole-school events. One teacher said, "These were very 
positive for ttiose kids wtiose parents participated! However, we still tiave not reached 
parents who are reluctant - those who were (as l(ids)/are sl<eptical or cynical about 
school environments. I don't thinl< we've changed very many parents' attitudes for the 
better" 

Quantitative Findings 

Many educational evaluators and researchers assert that evaluation questions 
should determine the methods used rather than the philosophical and theoretical 
position of the evaiuator (Howe & Eisenhart, 1990; Reichardt & Cook, 1979). Filstead 
(1979) suggests that a blending of the assets of both approaches throughout an 
evaluation is optimal. The evaiuator, unlike a researcher, often does not have the 
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luxury of being able to determine what project objectives will be the basis of an 
evaluation. She must use evaluation techniques which will determine whether project 
objectives have been attained. She must identify the stake holders in the educational 
context and the types of data they require for decision-making. And finally, the 
evaluator is compelled to make a judgment of the worth of an educational reform 
project within the educational setting rather than simply describe what has happened 
and why. 

The CLASS Project had student outcome (cognitive and affective) objectives which 
required quantitative data collection. Interpretations of comparisons of student scores 
on standardized tests and attitude scales were complicated by the school district 
reorganization which reduced the student population in the school by approximately 
one-half after the first year, the high transient rate of students, and the non-project 
curricular innovations being implemented in the school at the same time as the project. 
Because of these and other emergent problems, approaches to quantitative data 
analysis changed a few times during the project. 

Student Basic Skills. Student standardized test scores on the ITBS were not 
increased in a consistent manner across grade levels and across the three years of 
the project. Comparisons of ITBS test scores in reading and mathematics for the third 
grade students indicated no significant differences. Comparisons of ITBS test scores 
for fifth grade students across the three years of the project in reading indicated that 
fifth grade students in the second and third years of the project scored significantly 
higher than fifth grade students in the first year of the project. Comparisons of ITBS 
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Total Mathematics test scores for fifth grade students indicated that students in the 
second and third years of the project scored significantly higher than fifth grade 
students in the first year of the project. 

Results of teacher-constructed pretest/posttests for the four interdisciplinary 
units in year three indicated that student knowledge increased as a result of the units. 
This finding must be interpreted knowing that tests were not consistent across 
teachers and that approximately one-third of the teachers each quarter did not use a 
pretest/posttest system where student learning gains could be measured. 

Teachers' perspectives about the effect of CLASS on basic skills acquisition of 
students varied widely. Some teachers felt students' knowledge of basic skills had 
been radically improved and others indicated that they had reservations in some 
areas. Most teachers across grade levels felt student skills in reading and writing had 
increased as a result of the project. A small number of teachers expressed concerns 
about lack of covering basic skills and a few teachers felt lower performing students 
needed more structure than they were receiving. One teacher said, "The jur\f is stiH 
out on the effect on basic sMs. / sometimes worried that some basic s/f///s got 
skimmed over - Time may be needed to reaiiy judge." Another teacher said, "in basic 
sl<ills, some of our kids are riding higher ievei kids coat taiis. We're not emphasizing 
feas/c skiils enough. We are more project oriented. Lower performing kids need more 
structured approaches to basic skiiis with driiis and other Uiings. We provide that and 
remediate basic skiiis some. A iot depends on which teacher they have - on which 
extreme they work." Most teachers across grade levels felt students' skills in 
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identifying connections across disciplines was greatly enhanced by interdisciplinary 
unit instruction. 

Most parents wlio responded to the "parent report card" indicated that they feel 
basic subjects like reading, writing, science, mathematics, and social studies are either 
"satisfactory" or "excellent" at Fowler Drive. Comparatively few parents expressed any 
concerns about or suggestions for improving the curriculum. Only 7% of respondents 
felt that writing and mathematics instruction "needs improvement." 

Based on the low numbers (10 out of 24) of students successfully discontinued 
from the Extended Literacy Instruction (ELI) component, and the large amount of time 
required for instruction of a student by a teacher, the one-to-one instruction was not 
successful. About half of the teachers in the school felt that the ELI component was 
one of the most successful aspects of the project because of the degree to which 
knowledge of new reading strategies used in ELl improved their ability to teach and 
assess performance of all of their students in reading and writing. They felt they were 
continually increasing their understanding of the teaching-learning process relative to 
reading because a group of the faculty began meeting regularly to discuss current 
research and books in the area which they selected and all read prior to discussions. 

Student Attitudes. Data from the Brookover Student Sense of Academic Futility 
Questionnaire for the first two years of the project and data from the Coopersmith Self- 
Esteem Inventories used in year three of the project indicate that students' attitudes 
toward self and other class members were not significantly improved. There was very 
little change in student attitudes across years. Data on the Arlin-Hills Attitude Toward 
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Learning Process Survey indicate that students* attitudes toward learning did increase 
slightly in the third year, but did not increase to the 2.0 level which is defined as 
"positive as desirable" by the authors of the instrument. 

Project Conclusion 

Teachers, administrators, and parents at Fowler Drive Elementary School felt that 
the CLASS Project was successful. The project has not been validated, however, 
because members of the validation team felt that student standardized test scores did 
not demonstrate positive increases in student knowledge. Members of the validation 
team would have accepted alternative assessment data as "proof" of increased student 
learning, but none of the approaches attempted worked within a variety of contextual 
and educational limitations. Portfolio assessment took too much time and teacher 
anxiety was elevated because they felt they were not covering state and county 
mandated objectives adequately if they took time to do portfolio assessment as 
proposed. Performance assessment has not been developed enough for use across 
disciplines and grade levels. Teachers lacked knowledge of measurement needed to 
construct high quality and consistent pretests and postlests for teacher-constructed 
interdisciplinary units. 

Teachers at the school say that they feel they will be in a "state of school 
reform" for the continuation of their professional lives. They now realize that teaching 
is a process of continual learning, seeking the best methods for instruction, and taking 
the risk of trying different things to determine their effectiveness with children in 
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specific contexts. 



Discussion 

School reform at Fowier Drive Elementary School from 1989 to 1992 took the form 
of structural change and curricular change. More important than the specific changes 
in the three years, however, the process in which educators were engaged during that 
time of reform resulted in a change in the way teachers defined "self" and "non-self" as 
well as their roles as professional educators. They moved from the world of the 
classroom into being professional educators interested in the most current educational 
issues, theories, and debates in the literature and at professional conferences. 
Groups of the faculty traveled to other innovative schools to talk to educators and . 
observe ongoing educational reform. Some of the teachers began to research their 
own classrooms. Most teachers viewed the teaching-learning process differently than 
they had prior to the project. 

Teachers viewed themselves as "risk-takers" to try educational innovations which 
they determined might enhance the teaching-learning process. They placed a high 
value on this in their culture of schooling. They viewed faculties at other schools not 
engaged in some type of reform as stale and uninteresting. They were concerned that 
teachers in other schools in their county viewed them as "weird" or trying crazy things, 
but they were assured that their reform effort had merit as a large number of faculty 
representative groups from schools all over the nation visited the school to see how 
the implementation of innovations was working. They did not want to leave Fowler 
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Drive and go to schools where reform was not being addressed eventhough they were 
experiencing a high level of stress and exhaustion from the large number of 
collaborative meetings for curriculum planning and development, the increased time for 
interdisciplinary and cross-grade level teacher planning, the large amount of 
papenA^ork and documentation, the frustration of trying to identify and acquire outside 
resources for learning experiences, the time required to read and critique professional 
literature, and other activities. 

Students did not make significant gains on standardized tests, but teachers did not 
feel that they would. There were two major reasons for teachers* perceptions: 1) they 
did not feel that their approaches focusing on learning processes, learning 
demonstrated by performance, collaborative learning, and higher order thinking skills 
would affect student scores on standardized tests immediately; and 2) they decided as 
a faculty to stop teaching test-taking skills to students for three weeks prior to 
administration of standardized tests each year as had been done in the past. 

Some insights about school reform in this context seem to speak to school reform 
in all schools. School reform takes a long time and requires more than a typical two- 
year funded project. Measurement of the effects of school reform on student learning 
and determination of degiee of integration of innovation into existing structures and 
curricula generally require longitudinal evaluation beyond the boundaries of short-term 
funded projects. Teachers have more ownership in school reform if they help decide 
what changes are needed in their school. These teachers felt that curricula developed 
and selected by teachers to meet specific needs of their students would be far 
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superior to blanket adoption of curricula developed by external educators. School 
reform needs to be flexible so that changes can be made when teachers determine an 
innovative strategy is not effective for their students. Teachers need scheduled blocks 
of time in evei7 day for collaborative planning, examination of and discussion of ideas 
in educational research and theoretical literature, identification and evaluation of 
learning resources, and coordination of ways to share their experiences with teachers 
in other settings. There needs to be a person or small committee of persons in the 
school who coordinate and evaluate the reform effort systematically and in an ongoing 
manner to provide feedback to teachers. Documentation and student assessment 
which reflect more active learner approaches to learning need to be developed. 
School administrators need to be aware that all faculty will not implement innovations 
at the same rate and need to provide sharing and support systems among faculty. 

Conclusions 

The school reform project at Fowler Drive Elementary School was very complex 
and fluid. The reform included structural and organizational school components as 
well as curricular components. Teachers asserted that the most positive outcome of 
the school reform project v/as understanding that reform is an ongoing process and 
that a description of the process they used is the most exportable component of the 
reform to other schools. Teachers were envisioning changes, studying educational 
innovation issues and research literature, attending workshops, developing curricula, 
traveling to other schools in the nation to observe innovations, implementing 
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innovations in classrooms and the whole school, documenting the process, and 
evaluating effectiveness in an ongoing cyclical fashion throughout the project. This 
was all happening at the same time other major innovations were adopted in the 
school. Teachers and administrators were experiencing high levels of stress because 
of the lack of time to adequately complete any tasks. Additionally, major school 
district rezoning occurred at the end of the first year of the project which reduced the 
size of the school by almost half and many teachers were transferred to other schools. 
At the state level, requirements for standardized testing were changed after the second 
year of the project resulting in data which were availabie the first two years not being 
available for comparison in the final year of the project. 

Both process and product evaluation data were required by different stakeholders 
in the process. The funding agency wanted data on the effects on student learning as 
well as data describing the process and detailing the effects on teachers, 
administrators and parents. State and local administrators wanted standardized test 
score data because those were used to compare schools within the district and 
districts across the state. Teachers wanted documentation of tilings which enhanced 
the teaching-learning process as well as those things which were barriers. As Guba 
and Lincoln (1989) assert, stakeholders are the users of evaluation information that 
they seo as clearly responsive to the issues and concerns that they have. Policy 
decisions about educational practice based on both sets of data would be much more 
accurately informed about impact from multiple perspectives. 

Because of the complexity of educational reform and the diversity of interest of 
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stakeholders, both qualitative and quantitative data should be collected. Each type of 
data provides understanding about aspects of school reform efforts that the other can 
not. Both evaluation approaches are labor and time intensive and as Bogdan and 
Biklen (1992) suggest, one evaluator attempting to do both might produce a less than 
exemplary product on both. Collaborative evaluation teams with a qualitative and a 
quantitative evaluator are the most effective in determining the impact of educational 
reform in schools. As Smith and Heshusius (1986) point out, the l-wo approaches are 
based on different assumptions. By triangulating evaluators, each can base inquiry in 
the assumptions of the approach being used and the evaluation questions posed can 
be addressed with the appropriate methodology rather than the methodology driving 
the evaluation (Howe & Eisenhart, 1990). Evaluators can maximize use of their 
training, experience, ar^d philosophical and theoretical understandings of their 
preferred methodology when they work in collaborative qualitative and quantitative 
teams to evaluate school reform. The resulting evaluation is a more holistic 
understanding of the impact of implementation of innovation in the culture of 
schooling. 
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Figure 1. Brown Model of School Reform Process at Fowler Drive Elementary 
schooL 
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